Integrative multimodal image analysis using physical models for characterization of brain tumors in radiotherapy

Therapy failure with subsequent tumor progress is a common problem in radiotherapy of high grade glioma. Definition of treatment volumes with CT and MRI is limited due to uncertainties concerning tumor outlines. The goal of the presented work was to enable assessment of tumor physiology and prediction of progression patterns using multi-modal image analysis and thus, improve target delineation. Physiological imaging modalities, such as 18F-FET PET, diffusion and perfusion MRI were used to predict recurrence patterns. The Medical Imaging Interaction ToolKit together with own software implementation enabled side-by-side evaluation of all image modalities. These included tools for PET analysis and a module for voxel wise fitting of dynamic data with pharmacokinetic models. Robustness and accuracy of parameter estimates were studied on synthetic perfusion data. Parameter feasibility for progression prediction was investigated on DCE MRI and 18F-FET PET data. Using the developed software tools, a pipeline for prediction of tumor progression patterns based on multi-modal image classification with a random forest machine learning algorithm was established. Exemplary prediction analysis was applied on a small patient set for illustration of workflow functionality and classification results

MITK-ModelFit: A generic open-source framework for model fits and their exploration in medical imaging -- design, implementation and application on the example of DCE-MRI

Many medical imaging techniques utilize fitting approaches for quantitative parameter estimation and analysis. Common examples are pharmacokinetic modeling in DCE MRI/CT, ADC calculations and IVIM modeling in diffusion-weighted MRI and Z-spectra analysis in chemical exchange saturation transfer MRI. Most available software tools are limited to a special purpose and do not allow for own developments and extensions. Furthermore, they are mostly designed as stand-alone solutions using external frameworks and thus cannot be easily incorporated natively in the analysis workflow. We present a framework for medical image fitting tasks that is included in MITK, following a rigorous open-source, well-integrated and operating system independent policy. Software engineering-wise, the local models, the fitting infrastructure and the results representation are abstracted and thus can be easily adapted to any model fitting task on image data, independent of image modality or model. Several ready-to-use libraries for model fitting and use-cases, including fit evaluation and visualization, were implemented. Their embedding into MITK allows for easy data loading, pre- and post-processing and thus a natural inclusion of model fitting into an overarching workflow. As an example, we present a comprehensive set of plug-ins for the analysis of DCE MRI data, which we validated on existing and novel digital phantoms, yielding competitive deviations between fit and ground truth. Providing a very flexible environment, our software mainly addresses developers of medical imaging software that includes model fitting algorithms and tools. Additionally, the framework is of high interest to users in the domain of perfusion MRI, as it offers feature-rich, freely available, validated tools to perform pharmacokinetic analysis on DCE MRI data, with both interactive and automatized batch processing workflows.Comment: 31 pages, 11 figures URL: http://mitk.org/wiki/MITK-ModelFi

HDS-LEE Course on Hyperparameter Optimization

Part I: Theory - Basics of Hyperparameter Optimization - Exhausive Searches - Surrogate-based Optimization, Sequential Model-based Optimization and Bayesian Inference - Evolutionary Strategies Part II: Hands-on programming session "Hyperparameter Optimization for Improving Neural Networks" - Manual hyperparameter optimization of a Keras regression model - Machine-assisted optimization of the Keras model using Talos - Good practice guidelines for hyperparameter tuning

Feasibility and robustness of dynamic F-18-FET PET based tracer kinetic models applied to patients with recurrent high-grade glioma prior to carbon ion irradiation

The aim of this study was to analyze the robustness and diagnostic value of different compartment models for dynamic F-18-FET PET in recurrent high-grade glioma (HGG). Dynamic F-18-FET PET data of patients with recurrent WHO grade III (n:7) and WHO grade IV (n: 9) tumors undergoing re-irradiation with carbon ions were analyzed by voxelwise fitting of the time-activity curves with a simplified and an extended one-tissue compartment model (1TCM) and a two-tissue compartment model (2TCM), respectively. A simulation study was conducted to assess robustness and precision of the 2TCM. Parameter maps showed enhanced detail on tumor substructure. Neglecting the blood volume V-B in the 1TCM yields insufficient results. Parameter K-1 from both 1TCM and 2TCM showed correlation with overall patient survival after carbon ion irradiation (p = 0.043 and 0.036, respectively). The 2TCM yields realistic estimates for tumor blood volume, which was found to be significantly higher in WHO IV compared to WHO III (p = 0.031). Simulations on the 2TCM showed that K1 yields good accuracy and robustness while k(2) showed lowest stability of all parameters. The 1TCM provides the best compromise between parameter stability and model accuracy;however application of the 2TCM is still feasible and provides a more accurate representation of tracer-kinetics at the cost of reduced robustness. Detailed tracer kinetic analysis of F-18-FET PET with compartment models holds valuable information on tumor substructures and provides additional diagnostic and prognostic value

Feed-Forward Optimization With Delayed Feedback for Neural Networks

Backpropagation has long been criticized for being biologically implausible, relying on concepts that are not viable in natural learning processes. This paper proposes an alternative approach to solve two core issues, i.e., weight transport and update locking, for biological plausibility and computational efficiency. We introduce Feed-Forward with delayed Feedback (F$^3$), which improves upon prior work by utilizing delayed error information as a sample-wise scaling factor to approximate gradients more accurately. We find that F$^3$ reduces the gap in predictive performance between biologically plausible training algorithms and backpropagation by up to 96%. This demonstrates the applicability of biologically plausible training and opens up promising new avenues for low-energy training and parallelization

Deep-Learning-Based 3-D Surface Reconstruction—A Survey

In the last decade, deep learning (DL) has significantly impacted industry and science. Initially largely motivated by computer vision tasks in 2-D imagery, the focus has shifted toward 3-D data analysis. In particular, 3-D surface reconstruction, i.e., reconstructing a 3-D shape from sparse input, is of great interest to a large variety of application fields. DL-based approaches show promising quantitative and qualitative surface reconstruction performance compared to traditional computer vision and geometric algorithms. This survey provides a comprehensive overview of these DL-based methods for 3-D surface reconstruction. To this end, we will first discuss input data modalities, such as volumetric data, point clouds, and RGB, single-view, multiview, and depth images, along with corresponding acquisition technologies and common benchmark datasets. For practical purposes, we also discuss evaluation metrics enabling us to judge the reconstructive performance of different methods. The main part of the document will introduce a methodological taxonomy ranging from point- and mesh-based techniques to volumetric and implicit neural approaches. Recent research trends, both methodological and for applications, are highlighted, pointing toward future developments